1. Field of the Invention
This invention relates generally to camera stabilization systems and more particularly to an inexpensive lightweight handheld camera stabilization system employing distributed gyro rotation sensors to facilitate stabilization over a wide range of camera positions.
2. Description of the Related Art
Introduction: Advances in optical, video, and traditional photographic technologies have made high quality portable photographic equipment available to a growing number of film enthusiasts. As a result, motion pictures of increased quality and complexity are produced by professionals and enthusiasts alike. However, this quality evolution has exacerbated the well-known “jitter” problem when shooting with a hand held camera. Even when the operator tries to hold the camera steady during panning or translation, the transmission of uncontrolled operator motion to the camera results in unwanted camera jitter, detracting from the aesthetics of the resulting video product. Shooting from a moving, floating or airborne vehicle is even more difficult because of the uncontrolled vehicle motion. Turning or carrying a camera in a smooth glide requires a great deal of skill and experience, particularly in action scenes where the camera operator is walking, running, or riding in a vehicle to follow the subject of the film. In action situations, the already difficult task of holding a camera steady despite motions resulting from the operator's breathing, heart beat, and involuntary muscle movements, may be aggravated by uncontrolled environmental conditions. The resulting footage is often erratic, jerky, and visually unappealing. Motion disturbances, particularly angular disturbances transmitted from the operator to the camera, are the primary problem.
Mechanical Stabilizers: Some time ago, in U.S. Pat. Nos. 4,017,168 and Re 32,213, Brown disclosed a popular mechanical (non-electronic) camera stabilization device (the Steadicam®). The Steadicam rig uses a large counterweight mounted a significant distance from the camera that shifts the rig's center of mass to a handle that the operator then manipulates. Although popular, Brown's device is heavy, large and awkward, puts a strain on the operator, requires a long set up time, requires a cumbersome operator harness, and limits the range of camera movement. The operational smoothness of a Steadicam depends mainly on the operator because the operator must manually control the camera's orientation via a handle and must carefully keep the camera poised and balanced within the narrow range of system capabilities. This makes operation of and smooth slewing of the camera a function of operator strength and skill.
Rotating Flywheels: Other practitioners have proposed reaction wheel flywheels for camera stabilization. For example, in U.S. Pat. No. 4,774,589, Rowland discloses a stabilizer that measures the torque applied to the visual system about an axis and applies a reactive counter-torque created by accelerating an attached flywheel. An additional flywheel is needed on each axis, which adds significant weight to a handheld stabilizer and creates unwanted gyroscopic precession dynamics in the system. Stabilizers based on reaction wheels, such as Rowland's, do not isolate operator motion from the camera below 1 Hz, because correction torque is developed by accelerating the reaction flywheel. Because there are practical limits on how fast the flywheel can be spun up, there is a limitation on how long it can be continuously accelerated and used as a source of stabilizing counter-torque. Rowland's device permits only slow manual panning because his stabilizer attempts to suppress rapid panning as undesirable vibration. Rowland suggests that “not using gyroscopes is an advantage” but does not suggest how to avoid the disadvantages of the heavy counter-torquing flywheels.
In U.S. Pat. No. 6,730,049, Kalvert discloses a device for stabilizing tremors that fixes one or more gyroscopic flywheels running at a high, constant speed to a rigid splint for receiving a patient's hand, wrist and forearm to stabilize hand tremors. Similarly, a high-speed, flywheel can be attached to a camcorder, handheld camera, spotting scope or binocular to stabilize it in two or more axes. Kenyon Labs produces KS-2, KS-4, KS-6, KS-8 and KS-12 sealed dual counter-rotating brass or tungsten flywheels (“gyros”) that are spun by brushless motors at about 22,000 RPM in a bulky “hermetically sealed helium-filled housing” in an apparent effort to reduce high drag, heat dissipation and power consumption. One “gyro” mounted to a camera in line with the lens will resist motion in both pitch and yaw. Three such “gyros” may be mounted to a camera orthogonally to stabilize motion in more than two axes. The higher the moment of inertial of the flywheel and the faster it is spun, the more effective it is in stabilization. However a flywheel's moment of inertia is increased only by adding size and/or mass, and high speeds cause high drag—all disadvantageous for a lightweight, battery-powered handheld stabilizer. Kenyon Labs' camera stabilizer units require 26 continuous watts of power for 4 to 7 minutes to spin up, weigh up to 5 pounds or more, and are up to 6 inches long.
Lens-Only Stabilizers: Other practitioners propose camera image (lens-only) stabilization methods that rely on controlling the motion of a camera's lens alone, without considering the motion of the camera body. Image or lens-only stabilization has the disadvantage of very limited range of motion. No provision is made for panning over a wide range or for any automatic slewing. And, without relative position sensors and control logic, such systems are inherently unable to distinguish between intentional and unintentional operator motion—they recognize fast panning as undesirable vibration and act to suppress it. Because these methods operate only to actuate the lens vertically or horizontally, they cannot provide compensation for camera roll and they require lens position sensors for active control and end-of-range sensing. Several exemplary image or lens-only stabilization techniques are now discussed.
In U.S. Pat. No. 5,335,032, Onuki, et al. disclose such a lens-only stabilization apparatus that processes the output from an angular accelerometer in both a high-pass filter “for cutting the DC component of the angular acceleration signal” from the angular accelerometer and an integrator for “integrating the angular acceleration signal” to estimate “the angular velocity of vibration occurring in the lens.” An optical lens position sensor is employed to sense end-of-travel of the lens during panning.
Sato et al. [Sato et al., “Control Techniques for Optical Image Stabilization System,” IEEE Transactions on Consumer Electronics, Vol. 39, No. 3, August 1993] describe another such lens-only stabilization apparatus that rotates an optical “fluid prism” based on signals from an “angular velocity sensor” using “conventional PID (Proportional-Integral-Derivative)” control. The two angular velocity sensors used to detect pitching and yawing are not described. Disadvantageously, position and speed information are sensed and fed back from the prism unit to the controller. Also, fast panning apparently could be detected by the microprocessor as “unexpected fluctuation” and acted upon as undesirable vibration when it “distinguishes unexpected fluctuation from intentional panning and tilting”
Oshima et al. [Oshima et al., “VHS Camcorder with Electronic Image Stabilizer,” IEEE Transactions on Consumer Electronics, Vol. 35, No. 4, November 1989], describe yet another such lens-only stabilization method for discriminating between intentional and unintentional operator motion, using a microcomputer to perform “a time-domain statistical analysis of the detected angular velocity of the camera body” by making use of both a pair of piezoelectric vibratory angular rate gyros as well as two Hall-effect sensors to “detect the position of the lens relative to the camera body.” The requisite processing requires a dedicated microprocessor and associated software.
In U.S. Pat. No. 4,542,962, Strömberg describes a mechanical lens-only image stabilization system. Like Brown (the Steadicam) above, Strömberg relies on counterweights to buffer external vibration and help stabilize the image.
Similar commercially-available stabilized lenses are sold by Canon and Nikon. Nikon advertises that their “Vibration Reduction (VR) technology offers the equivalent of using a shutter speed 3 stops faster. In addition, active vibration mode selection possible for use in active situations such as a moving boat, car or plane and provides VR performance with Automatic detection of panning.” Panasonic offers a family of image stabilized digital cameras (Lumix Digital Cameras with MEGA Optical Image Stabilizer). Panasonic advertises that their Lumix DMC-FZ20K “has a built-in gyrosensor that detects any hand movement and relays a signal to a tiny microcomputer inside the camera, which instantly calculates the compensation needed. A linear motor then shifts the Optical Image Stabilizer lens as necessary to guide incoming light form the image straight to the CCD.” Minolta offers a digital camera that moves the image sensor instead of the lens to counteract camera shake. Minolta advertises that their DiMAGE A2 digital camera “features a CCD-shift mechanism to stabilize images by offsetting the shaking pattern of the user's hand. This gives unrivaled stability at up to 3 shutter speeds slower than on digital cameras without an Anti-Shake function.”
Exemplary of the limited stabilization ranges of such lens-only techniques is the gyro-stabilized binoculars produced and marketed as the Fujinon Techno-Stabi 14×40, for example, which provides only a ±5° “stabilization freedom” in a “lightweight” (three pound) package including two direct drive motors controlled by piezoelectric vibration sensors.
Electronic Image Stabilizers: Electronic image stabilizers operate to electronically translate or rotate a video image responsive to processed video signals detected in a video camera. Disadvantageously, because of the computational intensity of the requisite signal processing, such systems often update too slowly for practical real-time use and may be suitable only for software-based off-line processing of recorded video.
Kinugasa et al. [Kinugasa et al., “Electronic Image Stabilizer for Video Camera Use,” IEEE Transactions on Consumer Electronics, Vol. 36, No 3, August 1990], describe a method for horizontally or vertically moving a lens block only in response to image feature edge signal detections in the camera video signal. In effect, pan and tilt sensing are both located in camera video microprocessor. Disadvantageously, Kinugasa et al. achieved a stabilization rate of only 1 Hz and lens only control offers only a limited range of correction.
Murray et al. [Murray et al., “Motion Tracking with an Active Camera,” IEEE Transactions on Pattern Analysis and Machine Analysis, Vol. 16, No. 5, May 1994], describe a method for processing a video image sequence “using motion detection techniques requiring the detection and interpretation of feature edge motion from successive video frames. Murray et al. characterize the work as “methods of tracking a moving object in real time with a pan/tilt camera” but suggest no mechanical stabilization or control techniques.
In U.S. Pat. No. 6,002,431, Jung et al. disclose a digital image stabilization technique that pixel-shifts image frames electronically to create a more stable video image in a camcorder. There are no mechanical features to this technique. Similarly, in U.S. Pat. No. 5,253,071, MacKay discloses an all-electronic method and apparatus for determining the position of an image projected onto an oversized HDTV image sensor inside a video camera. Neither Jung et al. nor MacKay suggest any active mechanical stabilization techniques.
Gyro Sensor Stabilization: The camera stabilization art is replete with gyro sensor stabilized camera system proposals that rely on an integral three-axis gyro sensor to measure platform tilt, pan and roll from a single location coupled to the camera platform. Disadvantageously, the gyro sensor configuration used universally in the art collocates the three rotation sensors in a single package, which introduces several problems. Each sensor's sensitivity to rotation about a stationary axis varies according to the pivot rotation position about another axis. For example, the sensitivity of a yaw rate sensor mounted directly on the camera platform decreases from maximum to null as platform pitch angle increases from zero to 90 degrees. Also, at large pitch angles, the pitched yaw sensor introduces orthogonal rotation components into the signal used to control yaw motion. Practitioners in the art are keenly aware of this problem and have proposed various “band-aids” such as adding relative position encoders and gravity-level sensors and adding complex sine-cosine coordinate transformations to maintain constant control loop gain in the stabilizer control circuitry. As another example of these disadvantages, the usual closed-loop servo control systems are easily destabilized by mechanical resonances arising from mechanical decoupling and separation of the sensors from their respective actuators. Until now, practitioners generally have attempted to ease this problem by using mechanically stiff (heavy) gimbaled frames that are not suitable for lightweight, handheld stabilizers, which require lightweight frames that are unavoidably flexible.
For example, in U.S. Pat. No. 6,611,662, Grober discloses an autonomous, stabilized platform embodiment including an integral rate-sensor package for determining rotation rate about three perpendicular axes. The three angular rate sensors are collocated in a common package on the camera head base to determine the motion of, for example, the vehicle on which the stabilized platform is mounted. A high resolution encoder is attached to each actuator motor and feeds back the position of the camera support platform relative to the sensors on the head base. A second multi-axis sensor package, containing level sensors is fixed to the camera support platform. Because Grober does not provide means for aligning the camera mass centroid with the stabilizer system pitch and roll axes of rotation, large high-torque motors are required, which is very disadvantageous in light or maneuverable handheld embodiments. In fact, the most portable embodiment suggested by Grober is a heavy, stiff strapped-on operator rig. For example, Perfect Horizon offers a 2-axis commercial camera stabilization head for marine applications that weighs 27 pounds and measures 18×18×7½ inches, essentially as described in U.S. Pat. No. 6,718,130, also issued to Grober. Grober neither considers nor suggests changes to the angular rate sensor disposition to eliminate the disadvantageous angular interaction problem.
As another example, in U.S. Pat. No. 4,989,466, Goodman discloses a three-axis stabilized platform using servo motors driven by gyro rate sensors mounted on a common platform in a single “gyrostabilizer assembly.” Goodman notes that the roll and pitch signals disadvantageously interact as a function of pan angle, obliging him to add additional complexity to compensate for the fact that the sensors do not remain aligned with their respective actuators. Goodman employs a complex coordinate transformation technique where he applies the position sensor signals to a resolver and slip ring assembly where they are resolved through the pan angle into a coordinate system fixed with respect to the (camera) platform. Disadvantageously, while this transformation may fix the coordinates with respect to the platform, it does nothing to improve the poor sensor resolution near the unity sine/cosine angles. Goodman also suggests a fixed configuration that locates the (camera) platform mass centroid at the intersection of the three motor driven axes to eliminate unwanted torquing moments as a result of vehicle accelerations. Goodman neither considers nor suggests changes to the angular rate sensor disposition to eliminate the disadvantageous angular interaction problem.
As yet another example, in U.S. Pat. No. 6,263,160, Lewis discloses yet another three-axis stabilized platform for imaging devices such as motion picture and video cameras. Lewis employs three magnetic torque motors, an angular rate sensor embodied as a fiber optic gyro, and a capacitive angle-sensor array. The angular rate sensor array is collocated in a single package attached to the platform to detect rates of rotation in three dimensions relative to inertial coordinates. The capacitive angle sensors are deployed to sense the angular displacement between the base and the platform in three dimensions. Once again, the additional angle sensors employed by Lewis in his control system are required to address the same disadvantageous angular interaction problem described by Goodman above. Lewis neither considers nor suggests changes to the angular rate sensor disposition to eliminate the disadvantageous angular interaction problem.
Gyro Stabilized Airborne Systems: The military and aerospace camera stabilization art is also replete with gyro sensor stabilized camera system proposals that rely on an integral three-axis gyro sensor to measure platform tilt, pan and roll from a single position coupled to the camera platform. But these proposals tend to be much more expensive because of specialized environmental requirements and are generally ill-suited for application to mechanically-stabilized handheld stabilizer systems using lightweight, flexible frames.
For example, in U.S. Pat. No. 4,520,973, Clark et al. disclose a stabilized gimbal platform employing rate gyro stabilized gimbal rings for military infrared missile targeting systems. A bail gimbal is used with both pitch and yaw inner gimbals to provide low friction stabilization over a large angular pointing range. The bail itself is not mounted on its axis of rotation but rather is off-axis mounted on bearings for support and driven by an off-axis torque motor drive. One or two single-axis gyro rate sensors mounted together on a common platform may be used but Clark et al. neither consider nor suggest the use of a plurality of angular rate sensors distributed and individually fixed to their respective gimbals. Clark et al. neither consider nor suggest application of their military techniques to inexpensive handheld stabilizer systems using lightweight, flexible frames.
As another example, in U.S. Pat. No. 6,542,181, Houska, et al. disclose an aerial video camera system for installation on an airplane, including a video camera and recorder with an internal stabilization system. Houska, et al. teach that all high-performance aerial video camera systems require effective vibration compensation to eliminate the effects of high-frequency airplane and wind vibrations for smooth, jitter-free operation; that the current state of the art for effective compensation is possible only through the use of some form of gyro-stabilization, usually for the camera mount; and that gyro-stabilization is complex and expensive, with complete systems often costing more than the airplane itself Houska, et al. teaches the use of a foam sleeve to isolate the camera from high frequency airplane vibration.
As yet another example, in U.S. Pat. No. 5,897,223, Tritchew et al. discloses a three-axis stabilized platform including three magnetic torque motors and three orthogonal gyroscopes attached to a camera platform. Tritchew et al. suggest embodying the triple-gyro as an array of three fiber-optic gyros, but their control system embodiment also includes an inclinometer and an incremental shaft encoder capable of measuring only relative angle. The additional sensors and control system elements may be an attempt to overcome the same disadvantageous angular interaction problem described by Goodman above. Moreover, Tritchew et al. also include an outer sprung-shell vibration isolator to minimize the stabilization range required of their control system. Tritchew et al. neither consider nor suggest changes to the angular rate sensor disposition to eliminate the disadvantageous angular interaction problem. The Tritchew et al. patent is assigned to Wescam, who offer at http://www.wescam.com a number of stabilized, multi-spectral airborne imaging systems.
Commercial examples include Gyron Systems International, Ltd., http://www.gyron.com, offers a Dual Sensor Gimbal turret and camera, which measures 30×35×28 inches, for mounting outside of a helicopter. A two-channel fiber optic gyro package is used to stabilize the pan and tilt axes while the roll axis is stabilized by a quartz angular rate sensor. The outer pan and tilt gimbals are driven by direct-drive torque motors without gearing. The inner pan and tilt gimbals are driven by voice-coil actuators without gearing, to eliminate gear wear problems. The roll axis is driven by a torque motor driving through a ladder-chain. The angular rate sensors are grouped together on a common platform, as is universally done in the art, leading to the disadvantageous angular interaction problem described above. Gyron literature neither considers nor suggests application of their aeronautical techniques to mechanically-stabilized handheld stabilizer systems using lightweight, flexible frames.
Other commercial examples include Tyler Camera Systems, who offer side and nose mounted stabilized helicopter camera mounts for film and video formats at http://www.tylermount.com. They engineer and manufacture of custom stabilization platforms and appear to use the Kenyon Labs spinning flywheels described above in their Gyro-Stabilized Mounts for Helicopters” product line illustrated on their web site.
As a final commercial example, Crossbow Technology, http://www.xbow.com, is a supplier of inertial sensor systems for aviation, land, and marine applications, and markets its products for antenna and camera stabilization. A Crossbow vertical gyro or VG (VG400/VG700) system is strapped down to the aircraft and supplies roll and pitch data to the active gimbal. The gimbal uses the roll, pitch data and the gimbal's azimuth location relative to the aircraft in order to hold the camera at a constant angle relative to the horizon. Jitter reduction is simply measuring the activity of the camera and applying corrective signals to the motors in order to reduce the jitter. An IMU that measures rotation rate alone is typically used because no absolute position information is required. In digital video applications, jitter can be removed without the use of motors but sophisticated signal processing and rate/accelerometer data from an IMU. In the jitter reduction application, the Crossbow inertial system is mounted on the gimbal and frequently on the camera itself. Placing the sensor on the camera allows the controller to do closed loop control around a zero rate (null) sensor reading. In other words the sensor is actually measuring the error signal for the controller. The VG400 is found in many camera and antennae pointing systems. It uses MEMS technology, and it is the smallest of the above products. It reports roll, pitch information as well as 3-axes of angular rate data. The MEMS sensors are grouped together on a common platform, as is universally done in the art, leading to the disadvantageous angular interaction problem described above. Crossbow Technology literature neither considers nor suggests changes to the angular rate sensor disposition to eliminate the disadvantageous angular interaction problem.
As may be readily appreciated from these examples, the individual consumer selecting a camera platform stabilization system is limited to a variety of heavy, stiff commercial systems of moderate to high cost that require several heavy flywheel gyros each requiring a heavy power-pack or battery for spin-up. Because the military can afford the very high costs of more sophisticated systems, a military user may select from a broader range of devices employing very expensive and elaborate stabilization control systems in exchange for lower weight and power consumption. None of the available camera platform stabilization systems known in the art are suitable for applications requiring low cost, lightweight and low power consumption.
There is accordingly a clearly felt need in the art for a relatively inexpensive handheld camera stabilization system that is lightweight and flexible (maneuverable), stable over a wide range of angular positions and jitter frequencies (to DC). Such a stabilization system should also be suitable for smoothly simulating any desired camera motion responsive to a simple motion control signal transferred to the stabilizer controller. These unresolved problems and deficiencies are clearly felt in the art and are solved by this invention in the manner described below.